System and method of embedding rich media into text messages

ABSTRACT

While texting, a user is able access, share, and control rich media without leaving the texting application. The rich media are provided directly within the executing texting application. The texting application includes an embedded widget for controlling the rich media. Rich media includes, among other things, video clips, streaming audio, a map application, a movie-time application, a social movie-site application, a dynamically controllable image, or promotional media. Different mobile devices executing the texting applications communicate through a server that allows additional functionality, such as syncing the play of video clips and hosting and pushing the promotional media.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/955,572, filed Apr. 17, 2018, which is a continuation of U.S. patentapplication Ser. No. 14/079,957, filed Nov. 14, 2013 (now U.S. Pat. No.9,948,589), which claims the benefit of U.S. Provisional Application No.61/726,145, filed Nov. 14, 2012, all of which are incorporated byreference herein in their entirety for all purposes.

BACKGROUND

Texting is a commonly used feature of electronic devices such assmartphones. Today's devices do not allow a user to easily or seamlesslyaccess rich media or other interactive content while in the textingapplication. For example, when a user wishes to share rich media whiletexting, she must leave the texting application, navigate to the richmedia, copy the link to the rich media, return to the textingapplication, paste the link into a text message, and send the message toa recipient. Given the number of steps involved, many users are lesslikely to share rich media. FIG. 1 , for example, is a screen shot 100of a chat session after a user has performed most of these steps, rightbefore sending the message.

While the recipient is viewing the rich media, she cannot simultaneouslyexchange text messages with the sender. Instead, she must switch betweenthe texting application and the media playing application, making itimpossible to interact with the sender while both users are accessingthe rich media. In short, the user is severely limited in the tasks shecan perform while in the texting application.

SUMMARY

In accordance with invention, a user is able to view, listen to, orcontrol rich media during an active chat session. For example, a user isable to share a YouTube® video while texting, thus allowing her to textabout the video as she is watching it. She and a friend can sync theirviewing so that they are simultaneously watching the same parts of thevideo while texting each other, thus enhancing their viewing experience.While texting, the friends can watch or share any type of rich mediaincluding, but not limited to, videos, photos, and maps or promotionalcontent such as advertisements.

Also in accordance with the invention, within a chat session a user isable to activate media, such as by tapping an image to “bring it tolife,” and then expand it, share it, forward it, post it on Facebook®,or collapse it to minimize its screen footprint, all without leaving thechat session.

Users are also able to add features to a chat, such as by sharingcoupons during a chat session, communicate with “virtual friends” toreceive information about specialized topics from segregated databases,send and receive credit usable on the system hosting the chat session,automatically receive suggestions on topics related to the chat session,and customize interactive media, such as to provide interfaces toparticular Web sites, activities that can all be performed during chatsessions.

In one aspect of the invention, interactive content is embedded into atexting application. A server hosting content is queried to determineinformation about the content, information such as a title of thecontent, its duration, a preview clip or image, or other relevantinformation. A widget comprising rich media for interactively displayingand controlling the information is then generated on the fly. The widgetis then embedded into a texting application.

In one embodiment, multiple clients, and thus multiple textingapplications, are coupled through a server. The server performs certaintasks such as syncing content on multiple devices, notifying a senderwhen a recipient is playing rich media, and pushing advertisements intoa chat session, to name only a few such tasks. In one embodiment, userkeystrokes or gestures are automatically converted into emoticons andembedded within the text message.

In one embodiment, a plurality of selectable panes are concurrentlypresented on a device. Each pane corresponds to a chat session and has awidget embedded therein during the corresponding chat session. In oneembodiment, only one of the panes (chat sessions) is active at a time.

In a second aspect of the invention, a method of transmitting data to afirst device includes receiving a request from a first device for richmedia for embedding the rich media into a chat session executing on thefirst device and transmitting a packet containing the rich media to thefirst device. The first device is able to be any device, such as ahandheld device, a mobile phone, a smartphone, a personal computer, adesktop computer, a laptop computer, a personal digital assistant, atablet, an iPad®, a smart watch, or smart glasses.

In a third aspect of the invention, a method of organizing contactsincludes associating a virtual friend with a category and coordinating achat session between a user application executing on an electronicdevice and the virtual friend. In one embodiment, the method furtherincludes querying a specialized database for returning informationcorresponding to the category. The category includes, for example,sports, travel, movies, literature, or current events. In oneembodiment, the method also includes organizing the virtual friendwithin a list of virtual friends based on popularity or on sponsorshipfees.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a screen shot of a prior art chat session containing a link toa YouTube® video.

FIGS. 2A-C are screen shots of text sessions with different embeddedrich media in accordance with different embodiments of the invention.

FIG. 3 is a graphical representation showing types of rich media thatcan be embedded within text messages in accordance with embodiments ofthe invention.

FIG. 4 shows two mobile phones coupled through a host in accordance withone embodiment of the invention.

FIG. 5 is a flow chart of the steps for executing and controlling richmedia embedded within a chat session in accordance with one embodimentof the invention.

FIGS. 6-10 are screen shots of chat sessions each with differentembedded rich media in accordance with different embodiments of theinvention.

FIG. 11 is a screenshot of blocks showing multiple chat sessions inaccordance with one embodiment of the invention.

FIGS. 12-16 are screen shots of chat sessions each with differentembedded rich media in accordance with different embodiments of theinvention.

FIG. 17 is a screen shot of an interface for launching chat sessions inaccordance with one embodiment of the invention.

FIGS. 18A and 18B are screen shots illustrating ways to share rich mediaduring a chat session in accordance with embodiments of the invention.

FIGS. 19A and 19B are screen shots illustrating ways to present servicesand products during chat sessions in accordance with embodiments of theinvention.

FIG. 20 is a screen shot showing, among other things, threadscorresponding to virtual friends in accordance with one embodiment ofthe invention.

FIGS. 21A-D are screenshots showing dragging and dropping rich mediainto a chat session in accordance with one embodiment of the invention.

FIG. 22 is a screenshot showing an open API in accordance with oneembodiment of the invention.

DETAILED DESCRIPTION

In accordance with the principles of the invention, from within texting(chat) sessions, users are able to share rich media such as pictures,contact cards, YouTube® videos, Google® maps, and Spotify® songs. Thislist is exemplary only and does not limit this invention to only thesetypes of rich media. Users can seamlessly integrate and share picturesfrom the native Facebook® application, videos from the native YouTube®application, Web content from any browsing application, and any textingapplication from their smartphone. In one embodiment, the textapplication that supports the chat sessions includes an embedded widget.

As one example, the term “rich media” refers to objects that can becontrolled by user inputs, such as a video clip that can be started andstopped; a photograph that can be expanded, collapsed, shaded, colored,forwarded, shared, posted to a social networking site, or otherwisemanipulated; an interface for selecting and purchasing a product orservice; or a map that can be navigated.

As one example, an “embedded widget” appears to the user as anapplication executed within another application such that bothapplications appear to be executing simultaneously. In accordance withone embodiment of the invention, however, an embedded widget is anapplication that allows rich media to be formatted and displayed on adevice in a particular way. For example, when a user receives a URL torich media, such as by copying and pasting or by sharing it from anotherapplication, a module contacts a server that hosts the rich media. Theserver is queried for relevant, context-specific information about therich media. The information is then packaged, transmitted to the device,and then displayed within a texting application, such as describedherein. For example, when the URL is to a YouTube® video clip (e.g.,http://www.youtube.com/watch?v={videold}), the YouTube® server isqueried for relevant details of the video “videold.” The details caninclude the video name, a preview image, a description of the video, andthe playing time, to name only a few such details. This information isreceived on the device and, in accordance with the invention,graphically displayed on the device by a widget that is generated on thefly. The widget displays the relevant information as well as anycontext-specific action bar.

FIG. 2A is a screen shot 200 of a chat session with text message areas210A-C, rich media areas 205A-B, of which 205A contains an embeddedvideo clip, and a text input box 215. Users are able to chat using thetext message areas 210A-C while controlling the video clip 205A, such asby launching the video clip, pausing it, fast-forwading it, rewindingit, or stopping it. The users are able to perform these tasks withoutexiting the chat session. While a user is viewing the video clip, she isable to enter a text message in the text input box 215 to comment on thevideo clip or to suggest a similar video or content, to name only a fewsuch actions, without exiting from the chat sessions.

In one embodiment, the video clip is synced with the same video clipplaying on a second user's mobile phone. Because the two users arewatching the same portions of the video clip at the same time, they cancomment on the same portions in real time.

A user can have different chat sessions going on simultaneously. Forexample, a user can be concurrently chatting with Bill and Alice. Bytapping or otherwise selecting a text message area associated with Bill,the thread of all chat exchanges with Bill are activated. The user canscroll through all the messages and embedded rich media exchanged duringthe entire session with Bill. When that thread is activated, the usercan continue chatting with Bill. In one embodiment, the thread with Billis shown activated by having a bolder, richer color (e.g., 205A); theunactivated threads are shown as washed out (e.g., 205B). Those skilledin the art will recognize other ways to distinguish between activatedand unactivated threads.

FIG. 2B is a screen shot 210 of a chat session with text message areas220A-B, a rich media area 230 containing an interface to an“adopt-a-pet” interactive application, and a text input box 225. Theadopt-a-pet interactive application includes a user-selectable buttonfor adopting a pet. As in the screen shot of FIG. 2A, the user ispresented with rich media during the chat session. In the example ofFIG. 2B, the user is able to select and launch the adopt-a-petinteractive application at any time during the chat session.

FIG. 2C is a screen shot 250 of a chat session with a text input area260 and rich media area 250A. The rich media area 250A includes aGoogle® map 250A and a keyboard 250B. While viewing the Google® map, theuser is able to continue chatting with another user using the text inputarea 260. While chatting, the user is also able to control the richmedia. For example, the user can change the Google® map, such as byzooming in to or out from particular portions of the map, navigate todifferent portions of the map, or track her current location on the map,to name a few different actions. While viewing the map, the user can usethe chat session to communicate her location, arrival time, or otherrelevant information.

In accordance with one embodiment, the texting applications describedherein all include an embedded widget that displays and controls therich media. For example, in FIG. 2A, the embedded widget displays andreceives user input to control the video clip.

FIG. 3 is a schematic graphical representation 300 of some of the richmedia that can be embedded in a chat session using, for example, asoftware application 301 in accordance with different embodiments of theinvention. As shown in FIG. 3 , the illustrative rich media include, butare not limited to, Pandora®, Amazon.com®, Flixster™, Spotify®, andYouTube®, to name only a few such examples.

In one embodiment, the components of a system in accordance withembodiments of the invention include a client application, which runs ona mobile device, and a host application, which runs on a server. FIG. 4is a high-level diagram of two smartphones 401 and 410 coupled to eachother through a server 420, over a network 450. The smartphones 401 and410 each contains a copy of a client application 405, and the server 420contains a host application 425. The client application 405 includes atexting application with an embedded widget for executing rich media fordisplay and control within text messages. The network 450 can be anytype of network, such as the Internet, a wireless network, a wide areanetwork, a local area network, or any combination of these.

Those skilled in the art will recognize that, in one embodiment, theelements 401, 410, and 420 each includes computer-readable mediacontaining computer-executable instructions for performing thealgorithms described herein and a processor for executing thoseinstructions.

FIG. 5 is a flowchart of the steps 500 of a method performed by a clientapplication (e.g., element 405 in FIG. 4 ) in accordance with oneembodiment of the invention. In the step 501, a chat application islaunched on a mobile or other device such as a smartphone (e.g., 401 and410), such as by selecting a particular chat session. In the step 505,when embedded rich media is selected, such as by activating a videoclip, a corresponding interactive widget is launched. Preferably, thewidget is embedded in the chat session so that the chat session and richmedia are executing (or at least appear to the user to be executing)simultaneously. Some rich media accessed in accordance with theinvention have context-specific commands. For example, if the rich mediais a video clip (e.g., FIG. 2A), a play button is presented and, whenselected, causes the video clip to play. If the rich media displays auser interface to a pet adoption site (e.g., FIG. 2B), a select buttonis presented which, when selected, executes an application that causes apet to be adopted. In the step 510, any content-specific commands areexecuted.

The steps 500 are merely illustrative of one embodiment of theinvention. It will be appreciated that in other embodiments, a methodincludes different steps from those shown in FIG. 5 . In otherembodiments, some steps are added, some of the steps are deleted, andthe steps are performed in different orders.

As one example, a client in accordance with the invention (e.g., 405)includes the Android™ Gingerbread operating system (e.g., version 2.3 orabove) and communicates with a server (e.g., 420) using Google® protocolbuffers over HyperText Transfer Protocol (HTTP). The smartphones (e.g.,401 and 410) register with the server using Secure Sockets Layer (SSL),and transferred messages are signed with a secret token and Hash-BasedMessage Authentication Code (HMAC) authenticated. The server 420 of FIG.4 is capable of performing many other tasks, such as syncing content ondifferent smartphones, allowing users to simultaneously view the samecontent, etc.

In one embodiment, the server 420 comprises the Play! Framework, a Javaand Scala framework. The server 420 is deployed to a set of amazon EC2instances sitting behind a round-robin load balancer. The server 420contains a shared MYSQL database, such as an Amazon Relational DatabaseService instance or, alternatively, a multi-AZ deployment. The server420 uses a MemCached based cache cluster used separately or shared bymultiple servers. The server 420 stores photos and rich media using anAmazon S3 server. The server 420 is REST based, completely stateless,and can easily scale horizontally. The server 420 is able to send pushnotifications to devices using the Google® Cloud Messaging (GCM) serviceand employs a message queue to generate Web-page thumbnails andsmart-preview information. The server 420 sends registration ShortMessage Service (SMS) messages and is also able to host and push coupon,advertising, and other revenue-generating content to serve as rich mediain accordance with the principles of the invention.

It will be appreciated that this list of features is illustrative only.In different embodiments, the server 420 performs a subset thesefeatures, additional features, or a combination of any of them.

Bringing Content to Life in a Chat

Now that the architecture of a system in accordance with one embodimentof the invention has been discussed, additional examples illustratingthe principles of the invention are discussed. FIG. 6 is a screen shot600 of a chat session with a widget that includes a dynamicallycontrolled image 601 as rich media. During the chat session, a user isable to “bring the image to life,” or activate it, by tapping it. Theimage 601 has an associated context-specific action bar 605 with buttons605 a, 605 b, 605 c, and 605 d. When the image 601 is brought to life,the button 605 a is able to be tapped to expand the image 601 to fillthe entire screen and then later tapped to collapse the expanded image601 so that portions of other rich media are able to be seen; the button605 b is able to be tapped to post the image 601 on Facebook®; thebutton 605 c is able to be tapped to automatically share the image 601with selected friends; and the button 605 d is able to be tapped toforward the image 601 to selected friends. After tapping the button 605d, a list of the user's contacts (either in text, images, avatars, orother representations) is presented to the user, allowing him to selectwhich contacts to forward the image to. The image 601 can also be tappedto put it to sleep, so that it is no longer active.

The screen shot 600 also shows text areas 610A-C and inactive rich media620A-B, shown as “washed out” to indicate that they are inactive.

It will be appreciated that other types of rich media will have otheraction bars suited to the particular rich media. For example, inaddition to the actions in the action bar 605, a video clip can have anaction bar that also allows users to play, fast forward, or rewind thevideo clip. Rich media containing Yelp! content can contain a ratingelement that allows a user to rate a restaurant or other service. Richmedia containing Flickster content can contain a button to play a movietrailer and to buy tickets to the movie. These are but a few examples ofcontent-specific action bars that can be generated and used inaccordance with the invention.

Converting Links and Other Kinds of Data to Widgets

It will be appreciated that any kind of data shared from any application(e.g., html, text, link, picture) is able to be converted to aninteractive widget right inside the chat. Furthermore, the interactivewidgets display a context-sensitive action-bar with actions specific tothe displayed content. FIGS. 7 and 8 are screen shots illustrating thesefeatures.

FIG. 7 is a screen shot 700 of text message areas 705 and 711 of aninactive thread, text message areas 707 and 715 of an active thread,rich media areas 703 and 709 of inactive threads, and rich media areas701 and 713 of active threads.

In one embodiment, a user inputs a link to rich media into the textmessage area 715. After inputting the link (e.g.,http://www.youtube.com/watch?v=nv9NwKAjmt0&feature=g-logo), the richmedia is automatically embedded into a rich media area (e.g., 713). Thecontext-specific action bar 713 a is automatically embedded within therich media. In this example, the context-specific action bar 713 aincludes a title of the rich media (“Eton Style (Gangnam”)), a playbutton, a Facebook® button (f), a share button, and a forward button.Those skilled in the art will recognize other combinations of buttonsthat are able to be included in a context-specific action bar inaccordance with the invention.

In one embodiment, referring to FIGS. 4 and 7 , when the link is inputinto the text message area 715 and the user presses the Enter key, theserver 420 retrieves the rich media, formats the rich media to includethe context-specific action bar 713 a, and returns the formatted richmedia to the texting application 405 and the device 401, as shown inFIG. 7 . FIG. 8 is a screen shot 800 of similar chat session.

FIG. 9 is a screen shot 900 of a texting application in which the richmedia includes a streaming video. The icons 901-906 represent friendswith whom the user is chatting. By tapping one or more of the icons901-906, the user is able to perform certain actions. For example, theuser is able to receive a notification that a friend to whom he sent avideo clip has received the video clip and that the friend is currentlywatching it. The user is able to see what location in the video hisfriend is currently watching (e.g., 1 minute and 20 seconds into thevideo). The user is also able to sync his and his corresponding friend'swatching of the video. As the user is watching the video clip, he cansend a “sync” command to a server (e.g., element 425 in FIG. 4 ) totrigger syncing his position in the video with his friend's position.

In some embodiments, for privacy, a user is able to prevent others fromseeing that he is watching the rich media, from seeing his currentlocation in the rich media, or from allowing another to sync to hislocation in the rich media. The user can do this, for example, byselecting a “privacy” button or some similarly functioning button (notshown).

Examples

FIG. 10 is a screen shot 1000 of a texting application with an inactivetext message area 1005, an active text message area 1007, and rich mediaareas 1001 and 1003 with a keyboard 1009 in accordance with oneembodiment of the invention. The keyboard 1009 is configured toautomatically convert keystroke combinations into emoticons and embedthe emoticon into a text message. For example, pressing the “:” and “)”(colon and right parenthesis) keys, in that order, automatically insertsa smiley-face emoticon into the text message. Deleting the “)” keyreplaces the emoticon with the remaining character, a “:”. As anotherexample, pressing the “;” and “)” (semicolon and right parenthesis)keys, in that order, automatically inserts a blinking-face emoticon intothe text message. Other key combinations can be used to insert differentemoticons in accordance with embodiments of the invention.

FIGS. 11-16 show screenshots on devices in accordance with embodimentsof the invention. FIG. 11 is a screenshot 1100 on a device 1105 showingblocks or panes 1111-1114. Each of the blocks 1111-1114 corresponds to achat session between the user of the device 1105 and a different user.The blocks 1111-1114 are “stacked” one on top of another or “cascaded.”Each of the blocks 111-1114 includes, among other things, the name ofthe different user, the last text message exchanged between the user ofthe device 1105 and the different user, and the time the text messagewas sent. When one of the blocks 1111-1114 is selected, the entirethread of the chat session between the corresponding users is displayed.

FIG. 12 is a screenshot 1200 showing rich media 1210A-B, text messageareas 1220A-B, and a text input box 1230. FIG. 13 is a screenshot 1300showing interactive rich media 1310 and 1340, a text message area 1320,and a text input area 1330. The area 1310 includes a display area 1310Ashowing a video, a pause button 1310B, a slider 1310C for seeking at arandom location within the video, and a share button 1310D. The area1340 contains a keyboard.

FIG. 14 is a screenshot 1400 showing interactive rich media 1420 and1430, text message areas 1410, 1440, and 1450, and a text input area1460. FIG. 15 is a screenshot 1500 showing interactive rich media 1510,a text message area 1515, and a text input area 1520. The area 1510includes a display area 1510A showing a video, a play button 1510B and aslider 1510C for seeking to a random location in the video. The area1530 contains a keyboard. FIG. 16 is a screenshot 1600 showing the area1510 and blocks 1620, 1630, and 1640, with functionality similar to theblocks 1111-1114 in FIG. 11 .

FIG. 17 shows a screenshot 1700 of an interface for launchingapplications in accordance with the invention. The interface includesicons for launching Share 1710, Flipboard 1720, Gmail 1730, a textingapplication in accordance with one embodiment of the invention 1740,Messaging 1750, and Messenger 1760.

Adding Features to a Chat

In accordance with embodiments of the invention, a chat is customizableso that features can be added to it. In one embodiment, a chat sessionis customized so that rich media are shared between users during a chatsession. As one example, a user is presented with a menu of rich media,applications, such as Facebook®, or promotional materials, such ascoupons. Users can select these menu items during a chat session toshare with others participating in the chat session. For a fee, contentand service providers are able to have their content or servicespromoted in the menu. Higher-paying sponsors will have their content andservices displayed more prominently, such as by being placed higher inthe menu.

FIGS. 18A and 18B are screenshots illustrating two ways of sharing richmedia in accordance with one embodiment of the invention. FIG. 18A is ascreen shot of a menu (rich media) 1800 displayed after a user hasselected the plus (“+”) button within a chat session executing on aphone 1850. The menu 1800 contains sponsored items for Yelp® 1812,YouTube® 1813, Instagram® 1814, Google Chrome™ 1815, Google® maps 1816,Pinterest® 1817, and Spotify® 1818. When a user selects any of thesemenu items, the rich media is shared among users during the chatsession.

In one embodiment, a widget embedded in the chat session is customizedto generate the menu items in FIG. 18A by pressing the + button.Alternatively, the widget is customized to generate other menu items forsharing or to generate menu items by pressing a different button. Thoseskilled in the art will recognize other ways to add features inaccordance with the principles of the invention.

FIG. 18B is a screenshot 1850 showing a second way to share rich mediain accordance with one embodiment of the invention, directly from anyapplication on the phone 1850. In the embodiment of FIG. 18B, a textingapplication in accordance with one embodiment of the invention is one ofthe applications to be shared.

FIG. 19A is a screenshot 1900 in which the selections of rich mediainclude coupons, such as one for Mexican Grill 1901. FIG. 19B is ascreenshot on the device after the user has selected the option forMexican Grill during a chat session. After the coupon has been selectedduring the chat session, the coupon is automatically embedded within thechat session, without the user leaving the chat session. The coupon canthus be shared with other users taking part in the chat session. Thecoupons and other promotional rich media can be stored in a variety oflocations, such as on the host 420 in FIG. 4 , and later downloaded intoa texting application.

In other embodiments, a user gets a notification during a chat sessionthat a recipient of rich media such as a video is watching the video.This notification can be triggered when the rich media begins executingon the recipient's device.

Virtual Friends

The principles of the invention can be extended to include otherfeatures. For example, a user can add one or more virtual friends to thetexting application and later exchange messages with the virtualfriends. Virtual friends can be categorized. As some examples, a userhas a Major League Baseball (MLB) virtual friend “Chris” for baseball, avirtual friend “Sally” for movies, a virtual travel agent friend “Dani”for vacation traveling, a National Aeronautics and Space Administration(NASA) virtual friend “Ruth” for space travel, and a Sony representativevirtual friend “Ben” for music, to name only a few such virtual friends.A user is able to chat with these virtual friends and receiveinformation, including embedded rich media, but also non-embeddedinformation.

The virtual friends are able to text and share content such as text,photographs, audio, and video. A user is able to text a question aboutmajor league baseball to Chris and the system (e.g., 420, FIG. 4 ),using, for example, artificial intelligence applications, generates ananswer that is transmitted back to the user. Chris is able to send theuser a real-time text message that the user's team just scored, a videoof a scoring play that just occurred, rich content containing a “Buy”button for purchasing tickets to a game, or an interactive widget thatthe user is able to forward to another friend, to name only a few suchthings that Chris is able to do.

In one embodiment, virtual friends have a layer of virtual intelligence.The virtual friends have the intelligence to analyze a user's textinghistory (who or what he texted about), purchasing patterns, currentevents, and other information to determine what content may be ofinterest to the user. For example, if a user has bought baseball ticketswhen the Saint Louis Cardinals baseball team came to town, Chris is ableto notify the user that the Cardinals are in town next week. Chris isalso able to send rich content allowing the user to buy tickets to thegame.

In other embodiments, virtual friends are able to answer questions thatthe user posed. For example, a user shopping for a CD for her14-year-old nephew asks Ben (the Sony representative), “What is the mostpopular music for 14-year-old boy?” Ben responds with a list ofmusicians, rich media containing music clips, and a “Buy” button for CDscontaining the music clips.

Preferably, virtual friends have different knowledge bases. Each virtualfriend is connected to segmented content databases. Each databasecontains information in a specific category, allowing each virtualfriend to “specialize” in a particular area. In one embodiment, thevirtual friend applications and corresponding information databases arestored on one or more hosts such as the host 420 in FIG. 4 .

FIG. 20 is a screen shot 2000 showing, among other things, virtualfriends Matt 2002, for Major League Baseball, and Brigitte 2003, formake-up. Chat sessions are able to be triggered with these virtualfriends 2002 and 2003 and rich media embedded, such as described in theexamples above. Virtual friends are able to be organized (e.g, listed ina particular order) by popularity.

In some embodiments, the location of the list of virtual friends areable to be sold or sponsored. A market of friends is able to be run, andthe virtual friends are able to be added to the texting application. Thesystem suggests a variety of virtual friends and creates a marketplacefor users to create and suggest virtual friends to other users.

In other embodiments, the system determines friends that a user hastexted most often or most recently and offers them invitations toregister on the system.

In yet another embodiment, a virtual friends service is available bysubscription. For example, a user pays a weekly, monthly, or yearly feeto access the service.

System Credit

In another embodiment, a system provides an environment for generating,using, and exchanging credit. For example, money is connected to atexting application. Users can buy credit for use on the system. Usersare able to gain or win credit, which can be used to buy products orservices on the system, including other applications that execute on theuser's device, coffee at a local coffee shop, tickets on Fandango®,books on Amazon.com®, and products and services from Level App,Orderahead™, and Open Table®, to name a few examples. In addition, usersare able to send each other credit or products and services that theybought with the credit. Users can use their credit to unlock features ontheir device, such as more elegant emoticons that can be embedded intotheir text messages.

Auto Suggestions (while Writing a Message)

In another embodiment of the invention, suggestions are automaticallypresented to a user typing a text message. A user typing a text messagewill receive suggestions in real time. Preferably, the text isautomatically sent to a server (e.g., 420, FIG. 4 ), which analyzes thetext and determines one or more recommendations to make. The user isthen presented with cards or widgets of information to send to herfriends. As one example, a user texts her friend about a new movie. Theserver determines relevant content and sends the user a visual cardwidget that contains information about the movie or information about asimilar but better-rated movie, such as a trailer, show times, and a“Buy tickets” button. The user is able to send this visual card widgetto her friend. The suggestion is able to be forwarded with acontent-specific action bar, “Liked,” posted on Facebook®, or otherwiseshared as described in the above examples. Users are able to turn offthis “suggestion” feature if they choose.

Hybrid Mode

When a user is viewing any rich media that his friend has sent to him(the friend's location, a video, a Web site, a photo, a game, etc.)within “hybrid mode,” the rich media takes up only a part of the screen,allowing the user the ability to navigate to somewhere else in a textingapplication box in accordance with one embodiment of the invention(inbox, contacts list, another chat), while the rich media stays openfor the user to view.

Ability to Quickly Share Rich Media that a User is Viewing Via Drag andDrop/Tap

The user then is able to share the active rich media with someone elseby dragging and dropping or by tapping a button. The user is able todrop or push the rich media into another existing chat in the inbox,select a contact in the contacts list, or drop it directly into an openchat. FIGS. 21A-D illustrate sharing rich media this way in accordancewith embodiments of the invention.

FIG. 21A is a screenshot 2100A showing rich media (a video) 2110displayed during a chat session 2120. FIG. 21B is a screenshot 2100B ofthe same chat session, showing a destination 2120A within the chatsession to move the video 2110. FIG. 21C is a screenshot 2100C duringthe same chat session after the video 2110 has been tapped, therebysliding a panel 2110′ over the video. FIG. 21D is a screenshot 2100Dafter the video 2110 has been dropped and dragged at the area 2120A.

Ability to Quickly Forward a Message/Media Via Drag and Drop

The user is able to long-press a message or any other content in thechat page. The message or piece of content becomes selected as long asthe user does not leave it. The message or the content is able to bedragged around under the user's finger. If the content is moved towardsthe edges of the device screen, other screens of the application, suchas the list of conversations, are revealed.

The user is able to drag an item above different drop-targets (e.g., aconversation). When content is dragged so that it hovers over a specificdrop-target, that drop-target becomes selected. Raising a finger whilesuch a drop-target is selected will trigger forwarding of the content tothe selected drop-target.

Adding Gamification to Texting

In another aspect of the invention, gamification techniques are appliedto texting. Recognition and benefits are given users depending on howactive they are using the texting application. For example, a user getscredits, such as discussed above, based on the number of messages theysent using the system, the number of friends they have invited to thesystem, the number of YouTube® videos they shared using the system, thenumber of Facebook® postings they made, the number of friends to whomthey forwarded information using the system, or any combination of theseand other actions they took using a system in accordance with theinvention.

As one example, after a user has exchanged 1,000 messages with a friendusing a system in accordance with the invention, the user, the friend,or both are rewarded with a pleasing animation and accompanying videoclip. As another example, a user who has shared a particular video clipwith a pre-determined number of friends using the same system isrewarded with 20 credits for purchasing content, goods, and services,such as described above. A user is also able to purchase pre-paidcredit. Users are able to use credit to purchase content and servicesrelated to systems in accordance with the present invention, such as funapplications, badges, animated cards, better emoticons. A user is ableto purchase features on the system that were previously blocked to him.Users are also able to purchase content and services unrelated to thesesystems, such as e-cards for Starbucks Coffee® or new phones. Users cansend and receive this credit instantly, such as in embedded content in atexting application described herein.

Texting API

In another aspect, an “open” application programming interface (API) isprovided to users, allowing them to present their content to other usersof systems in accordance with embodiments of the invention. The APIallows users to determine what rich content is presented, how it ispresented, and what actions are able to be taken on the content. As someexamples, a user determines the content-specific action bar, allowingsuch actions as play, forward, zoom in, copy, “Post,” and “Like,” toname only a few such actions.

As one example, a user calls certain functions in the API to generate anaction bar tailored to access the United States Patent and TrademarkOffice (USPTO) Web site. Once these functions are called through theAPI, a widget is generated, with an action bar that allows a userreceiving the widget within a text message to search for issued patentsand patent applications at the USPTO, download documents from the USPTO,and get the status of specified patent applications, to name only a fewpossible actions. FIG. 22 shows a screenshot 2200 illustrating thisexample. The screenshot 2200 shows a text area 2201 and a rich media2205 that includes an input area 2205A for entering a patent numbers, a“Search” button 2205B, a “Download” button 2205C, and a “Get Status”button 2205D for performing associated functions in the USPTO Web site.

For any applications for which a particular API has not been provided, ageneric API is able to be provided. The generic API is able, forexample, to give a thumbnail image of a Web page.

In still other embodiments, when a link is copied and pasted, such asdescribed above in FIG. 7 , the link is automatically pasted as ashortened URL. This decreases the chances of a link “breaking,” as oftenhappens with longer links. In other embodiments, the screen is splitinto an application section and a chat section. Both the applicationsand the chats can be flipped. Chat logs are able to automatically bedivided into conversations.

In operation, a texting application contains a widget used to embed richmedia within the texting application. A user is able to watch andcontrol the rich media directly within the texting application. The useris able to collapse the rich media to simultaneously view different richmedia on his phone. He is able to activate the rich media and a chatsession associated with each one. Preferably, the rich media has acontext-specific action bar that allows the user to, among other things,expand, collapse, play, share, post, or forward the rich media. Aftersending rich media to a friend, the user is able to monitor his friend'sviewing or listening position in the media, and he is able to sync hisposition to that of his friend's.

Different embodiments of the invention are directed to automaticallysharing information, using, for example, robots and to automaticallyadding content such as cards and widgets; customizing work; tailoringcontent; and providing access to the system, such as through APIs.

In one embodiment, user devices (e.g., 401 and 410) and the server 420each contains computer-readable media for performing the algorithmsdescribed herein and a processor for executing those algorithms.

It will be appreciated that the embodiments described above areillustrative only and can be modified in many ways in accordance withthe principles of the invention. As one example, while some of theexamples describe using smartphones, it will be appreciated that anydevice connected to the Internet can be used in accordance with theprinciples of the invention. For example, embodiments of the inventionare also able to be used on any handheld device, mobile phones, personalcomputers, desktop computers, laptop computers, tablets, personaldigital assistants, iPads®, smart watches, and smart glasses, such asGoogle® smart glasses, to name only a few such devices.

It will also be appreciated that the embodiments can be combined in manydifferent ways to form systems with any combination of featuresdiscussed herein. For example, all of the features described herein areable to be included on a single device or distributed across multipledevices. It will be apparent to those skilled in the art that othervarious modifications may be made to the embodiments without departingfrom the spirit and scope of the invention as defined by the appendedclaims.

What is claimed is:
 1. A computer-implemented method comprising: adding,by a chat application on an electronic device of a first user, acategorized virtual friend as a virtual friend of the first user in thechat application; sending a user query to a server; accessing, by thecategorized virtual friend, a texting history associated with the firstuser to a second user; receiving, at the electronic device, a responsemessage based on the texting history from the categorized virtualfriend; executing, by the electronic device, code to display an embeddedwidget that is embedded within the chat application, wherein theembedded widget includes a video related to the response message;receiving user input to move a representation of the video into aconversation in a conversation portion of the chat application with oneor more contacts of the first user; responsive to receiving the userinput, forwarding the video from the electronic device to a number ofcontacts that are part of the conversation; responsive to the number ofother users exceeding a pre-determined number, generating a credit topurchase content or services within the chat application; and responsiveto purchase of the content using the credit, enabling the first user toaccess a feature of the chat application that is unavailable prior tothe purchase.
 2. The method of claim 1, wherein the texting historyincludes purchasing patterns and current events .
 3. The method of claim2, wherein the categorized virtual friend is made available via asubscription service and further comprising: receiving a payment via thechat application for the subscription service.
 4. The method of claim 3,wherein the credit is further based on at least one selected from thegroup of exceeding a threshold number of sent messages using the chatapplication, exceeding a threshold number of friends invited to the chatapplication, exceeding a threshold number of posts made in the chatapplication, and exceeding a threshold number of pieces of contentforwarded using the chat application.
 5. The method of claim 1, whereinthe user input to move the video includes a long-press of the video anda drag-and-drop motion to move the video into the conversation.
 6. Themethod of claim 1, wherein the response message includes a link to thevideo and further comprising: converting the link into the embeddedwidget with a context-sensitive action-bar with actions specific to thevideo.
 7. The method of claim 1, wherein the chat application is a firstchat application and further comprising: sending, by the chatapplication, based on a request from the first user, the embedded widgetto the second user associated with a second chat application, whereinthe embedded widget includes the video.
 8. A non-transitorycomputer-readable medium with instructions stored thereon that, whenexecuted by a processor of an electronic device, cause the processor toperform operations, the operations comprising: adding, by a chatapplication on the electronic device of a first user, a categorizedvirtual friend as a virtual friend of the first user in the chatapplication; sending a user query to a server; accessing, by thecategorized virtual friend, a texting history associated with the firstuser to a second user; receiving, at the electronic device, a responsemessage based on the texting history from the categorized virtualfriend; executing by the electronic device, code to display an embeddedwidget that is embedded within the chat application, wherein theembedded widget includes a video related to the response message;receiving user input to move a representation of the video into aconversation in a conversation portion of the chat application with oneor more contacts of the first user; responsive to receiving the userinput, forwarding the video from the electronic device to a number ofcontacts that are part of the conversation; responsive to the number ofother users exceeding a pre-determined number, generating a credit topurchase content or services; and responsive to purchase of the contentusing the credit, enabling the first user to access a feature of thechat application that is unavailable prior to the purchase.
 9. Thecomputer-readable medium of claim 8, wherein the texting historyincludes purchasing patterns and current events.
 10. Thecomputer-readable medium of claim 9, wherein the categorized virtualfriend is made available via a subscription service and the operationsfurther comprise receiving a payment via the chat application for thesubscription service.
 11. The computer-readable medium of claim 10,wherein the credit is further based on at least one selected from thegroup of exceeding a threshold number of sent messages using the chatapplication, exceeding a threshold number of friends invited to the chatapplication, exceeding a threshold number of posts made in the chatapplication, and [[or]] exceeding a threshold number of pieces ofcontent forwarded using the chat application.
 12. The computer-readablemedium of claim 8, wherein the purchase is for a product for anotherapplication that operates on the electronic device.
 13. Thecomputer-readable medium of claim 8, wherein the response messageincludes a link to the video and the operations further comprise:converting the link into the embedded widget with a context-sensitiveaction-bar with actions specific to the video.
 14. The computer-readablemedium of claim 8, wherein the chat application is a first chatapplication and the operations further comprise: sending, by the chatapplication, based on a request from the first user, the embedded widgetto the second user associated with a second chat application, whereinthe embedded widget includes the video.
 15. An electronic devicecomprising: one or more processors; and a memory, coupled to the one ormore processors, that stores instructions that, when executed by the oneor more processors cause the one or more processors to performoperations comprising: adding, by a chat application on the electronicdevice of a first user, a categorized virtual friend as a virtual friendof the first user in the chat application; sending a user query to aserver; accessing, by the categorized virtual friend, a texting historyassociated with the first user to a second user; receiving, at theelectronic device, a response message from the server; executing by theelectronic device, code to display an embedded widget that is embeddedwithin the chat application, wherein the embedded widget includes avideo related to the response message; receiving user input to move arepresentation of the video into a conversation in a conversationportion of the chat application with one or more contacts of the firstuser; responsive to receiving the user input, forwarding the video fromthe electronic device to a number of contacts that are part of theconversation; responsive to the number of other users exceeding apre-determined number, generating a credit to purchase content orservices within the chat application; and responsive to purchase of thecontent using the credit, enabling the first user to access a feature ofthe chat application that is unavailable prior to the purchase.
 16. Theelectronic device of claim 15, wherein the texting history includespurchasing patterns and current events.
 17. The electronic device ofclaim 16, wherein the categorized virtual friend is made available via asubscription service and the operations further comprise: receiving apayment via the chat application for the subscription service.
 18. Theelectronic device of claim 17, wherein the credit is further based on atleast one selected from the group of exceeding a threshold number ofsent messages using the chat application, exceeding a threshold numberof friends invited to the chat application, exceeding a threshold numberof posts made in the chat application, and exceeding a threshold numberof pieces of content forwarded using the chat application.
 19. Theelectronic device of claim 15, wherein the credit is for a badge or ananimated card.
 20. The electronic device of claim 15, wherein theresponse message includes a link to the video and the operations furthercomprise: converting the link into the embedded widget with acontext-sensitive action-bar with actions specific to the video.